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DETAILED ACTION 

1 . This following Office action is in response to communications filed on 
10/5/2010. Claims 1, 5, 7-9, and 11 are currently pending within this application. 

Response to Arguments 

2. The Applicant's arguments regarding the claims have been fully 
considered but are moot in view of new ground(s) of rejection necessitated by 
Applicant's amendments to the claims. 

Claim Rejections - 35 USC §103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

4. Claims 1 and 11 are rejected under 35 U.S.C 103 as being 
unpatentable over Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] in 
view of Freeman (US PGPub 2005/0280714) [hereafter Freeman] and Ono 
(US Patent 7,456,874) [hereafter Ono]. 

5. As to claim 1 , Nonaka teaches a multi-eye imaging apparatus (Fig. 1 ) that 
comprises a plurality of imaging systems including a first imaging system (lens, 
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1 1 , CCD 1 2, and actuator 21 ) and a second imaging system (lens 1 6 and CCD 
17), each imaging system including an optical system (Hand 16) and an imaging 
element (12 and 17) and having different optical axes (as shown in Fig. 1), 
wherein the first imaging system includes a shifter (actuator 21) that changes a 
first relative positional relationship between an image of a subject (110) formed 
on the imaging element (12) of the first imaging system and the imaging element 
of the first imaging system during a time-series image capture (camera operation 
of Fig. 3 where images 10-11 are captured in continuous time-series during 
exposure of the CCD of the first imaging system), and the second imaging 
system (16 and 17) has a second relative positional relationship between an 
image of a subject (1 1 0) formed on the imaging element (1 7) of the second 
imaging system and the imaging element of the second imaging system that is 
fixed during time-series image capture (Col. 3, 29-53, Col. 4, 47-59), an image 
memory (recording section 15 and image memory 19a) that accumulates a 
plurality of frames of image information captured in the time series from the first 
and second imaging systems (image frames 10-11 captured by the second 
imaging system are placed within image memory 19a and recorded images 
captured by the first imaging system are placed within recording section 15 
during the camera imaging operations shown in Fig. 3), a camera shake 
detection section (19) that obtains a shake amount (X1-X3 shown in Fig. 2) by 
comparing the plurality of frames of image information accumulated in the image 
memory captured in the time series and comprises a comparison section (19b), 
moving direction calculation section (19c), and a moving amount calculation 
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section (19d) (Col. 4, 9-46, Col. 6, 26-34), a parallax amount obtaining unit 
(calculation control section 25) that obtains a magnitude of a parallax from 
images captured by first and second various imaging systems (Col. 7, 63-67, Col. 
8, 1-6), an optimal image selecting unit (calculation control section 25) that 
selects a combination of image information from the plurality of frames of image 
information in the image memory, captured in the time series from the first 
imaging system and the second imaging system (Col. 4, 19-21, 26-39, the 
calculation control section selects images 10-11 captured during the time-series 
and stored within the image memory 19a part of the image memory to be 
combined), and an image combining unit (image processing/forming section 22) 
configured to combine the selected combination of the image information (Col. 6, 
26-34, the image processing/forming section synthesizes the image signals 
captured by the second imaging system and selected by the calculation control 
section stored within the image memory upon one another (shown in Fig. 5, 24b), 
corresponding to combining the plurality of selected frames of the image 
memory). 

It is however noted that Nonaka fails to particularly disclose a shifter that 
changes a relative positional relationship in a fixed change amount between an 
image formed on an imaging element and the imaging element, and an optimal 
image selecting unit that selects a combination of image information so that a 
resolution of a combined image information is higher than that of each of the 
plurality of frames of image information. 



Application/Control Number: 10/597,794 Page 5 

Art Unit: 2622 

On the other hand, Freeman discloses an imaging apparatus (Fig. 1, 
image enhancement system 100) that comprises a shifter (actuator 122 and 
optical compensator 108) that changes a first relative positional relationship in a 
fixed change amount (1 /1 0 or 1 /1 00 of a pixel length or width or any similar value 
to achieve a desired sub-pixel offset) between an image (102) formed on an 
imaging element (photodetector 104) and the imaging element (104) during a 
time-series image capture (Page 2, 0029, Page 3, 0030-0031, Page 4, 0038, as 
shown in Figs. 3A-3D, an actuator controls the movement of an optical 
compensator during image capture in order to change the positional relationship 
between an image 202 formed on the pixel array 200 and the pixel array, shown 
in transition from Fig. 3A to Fig. 3B for example, by a fixed amount such as 1/10 
or 1/1 10 of a pixel in the X or Y directions), and an optimal image selecting unit 
(resolution enhancement system 124) that selects a combination of image 
information from a plurality of frames of image information in an image memory 
(storage medium of system 124) captured in the time series so that a resolution 
of a combined image information is higher than that of each of the plurality of 
frames of image information (Page 3, 0031 , Page 5, 0046, 0048-0049, images 
captured during an imaging operation/time-series and stored within the storage 
medium of the system 124 are selected for combination in order to create an 
enhanced resolution image from the captured frames of image information), and 
an image combining unit (super resolution image processor 302) configured to 
combine the selected combination of image information (Page 4, 0040, Page 5, 



Application/Control Number: 10/597,794 Page 6 

Art Unit: 2622 

0048, 0052, the selected images are provided to and combined within the super 
resolution image processor in order to form an enhanced resolution image). 

It would have been obvious to one having ordinary skill in the art at the 
time of invention to include a shifter that changes a relative positional relationship 
in a fixed change amount between an image formed on an imaging element and 
the imaging element and an optimal image selecting unit that selects a 
combination of image information so that a resolution of a combined image 
information is higher than that of each of the plurality of frames of image 
information as taught by Freeman with the multi-eye imaging apparatus of 
Nonaka because the prior art are directed towards imaging devices equipped to 
change the positional relationship between an image formed on an imaging 
element and the imaging element and combine the shifted images and because 
all the claimed limitations are disclosed within the cited prior art and would 
enable the multi-eye imaging apparatus of Nonaka to form an enhanced 
resolution image by controlling the shifting amount of the position of the image on 
the imaging element by a particular amount providing continuity between the 
images and combining images determined by a selecting unit to provide a quality 
enhanced resolution image. 

Additionally, it is noted that the combination of the Nonaka and Freeman 
references fail to particularly disclose a parallax amount obtaining unit that 
obtains a magnitude of parallax from image stored within an image memory. 

On the other hand, Ono discloses a multi-eye imaging apparatus (Fig. 1) 
that comprises a parallax amount obtaining unit (control unit 50, depth 
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information extractor 62) that obtains a magnitude of parallax from images 
(parallactic images) stored within an image memory (memory 40) (Col. 3, 55-59, 
65-67, Col. 4, 1-17, 30-40, Col. 6, 56-67, Col. 7, 1-6, parallactic image data 
comprising of a plurality of images of a subject captured at various viewpoints are 
stored within an image memory and supplied to a control unit and a depth 
information extractor thereof where the parallactic images are analyzed by the 
depth information extractor in order to obtain a parallax amount). 

It would have been obvious to one having ordinary skill in the art at the 
time of invention to include a parallax amount obtaining unit that obtains a 
magnitude of parallax from image stored within an image memory as taught by 
Ono with the above combination of Nonaka and Freeman because the prior art 
are directed towards imaging devices that detect an amount of parallax from 
image information and because all the claimed limitations are disclosed within 
the prior art and would enable the multi-eye imaging apparatus of Nonaka and 
Freeman to obtain a parallax amount from the images stored within the image 
memory thus enabling information regarding the parallax between the stored 
images to be recognized and used to dictate subsequent combination or 
processing operations performed by the imaging apparatus. 

6. As to claim 1 1 , Freeman discloses the shifter (actuator 1 22 and optical 
compensator 108) changes the first relative positional relationship using only the 
fixed change amount (Page 3, 0031 , Page 5, 0048-0049, only the fixed change 
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amount that provides the desired sub-pixel offset of the image upon the sensor 
array is used during the imaging process). 

7. Claim 5 is rejected under 35 U.S.C 103 as being unpatentable over 
Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] and Freeman (US 
PGPub 2005/0280714) [hereafter Freeman] and Ono (US Patent 7,456,874) 
[hereafter Ono], as applied to claim 1, in further view of Yamasaki (US 
PGPub 2003/0071905) [hereafter Yamasaki] and Kawahara (US Patent 
7,095,001) [hereafter Kawahara]. 

8. As to claim 5, it is noted that the combination of Nonaka, Freeman, and 
Ono fails to particularly disclose the image combining unit corrects and combines 
the images based on the parallax amount obtained by the parallax amount 
obtaining unit. 

On the other hand, Yamasaki discloses an image processing method (Fig. 
10) that includes selecting image information (Fig. 9) captured by an image 
sensing unit (Figs. 2-7) to be used in the combination of the image information 
(performed at s310) based on the amount of parallax calculated (performed at 
s302) (Page 3, 0057, -0058, Page 4, 0080, Page 5, 0082-0084, 0088-0092, Page 
6, 0093-0094, 0099, the amount of parallax between captured images is 
calculated and based on the result, the images are modified/shifted in order to 
compensate for the parallax and the subsequent shifted image data are then 
combined to form a single composite image). 
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It would have been obvious to one having ordinary skill in the art at the 
time of invention to correct and combine captured and stored images based on 
an obtained parallax amount as taught by Yamasaki with the multi-eye imaging 
apparatus of Nonaka, modified with the teachings of Freeman and Ono, because 
the prior art are directed towards imaging apparatuses that combine captured 
image information wherein the image information have determined parallax 
amounts and because all the claimed limitations are disclosed within the cited 
prior art and would enable the device of Nonaka to select specific stored image 
information for combination with each other to produce an image with increased 
resolution through the synthesizing of the images, performed by the forming 
section 22, and choose the respective images for combination by taking into 
account the parallax amount due to the images being formed on the various 
image sensors with varying optical axes increasing the accuracy of the selection 
of images best suited for combination. 

Also, it is noted that the combination of the Nonaka, Freeman, Ono, and 
Yamasaki references fail to particularly disclose the image combining unit 
corrects and combines the images based on the shake amount obtained by the 
shake amount obtaining unit. 

On the other hand, Kawahara discloses an image sensing apparatus (Fig. 
10) that performs image blur correction and includes an optimal image selecting 
means (coordinate converter 173) for selecting image information which is used 
in the combination of the image combining means (image combiner 174) from 
image information captured by an imaging system (image sensing device 161) 
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accumulated within an image memory (171) based on a shake amount obtained 
by a shake amount , Col. 6, 1 -obtaining means (shift detector 172) (Col. 2, 38-40, 
Col. 3, 26-39, Col. 4, 3-13, 39-44, 60-67, Col. 5, 33-47, 57-67, Col. 6, 1-20, the 
coordinate converter selects image information to be combined by the image 
combiner for each image stored within the image memory by shifting coordinates 
of a feature point within the images to have the same values wherein the 
extraction/designation of features points and corresponding shifting of the image 
data are performed based on the amount of shift/difference detected by the shift 
detector of the extracted feature point, caused by camera shake, detected 
through image comparison). 

It would have been obvious to one having ordinary skill in the art at the 
time of invention to correct and combine captured and stored images based on a 
shake amount obtained by a shake amount obtaining unit as taught by Kawahara 
with the multi-eye imaging apparatus of Nonaka, modified with the teachings of 
Freeman, Ono, and Yamasaki, because the prior art are directed towards 
imaging apparatuses that combine captured image information where the image 
information contain determined shake amounts and because the claimed 
limitations are disclosed within the prior art and would enable the device of 
Nonaka to select specific stored image information for combination with each 
other to produce an image with increased resolution through the synthesizing of 
the images, performed by the forming section 22, that is additionally unaffected 
by blurring effects caused by camera shake based on the amount of shake 
detected by the shake detecting section of Nonaka. 
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9. Claims 7-8 are rejected under 35 U.S.C 103 as being unpatentable 
over Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] and Freeman 
(US PGPub 2005/0280714) [hereafter Freeman] and Ono (US Patent 
7,456,874) [hereafter Ono], as applied to claim 1, in further view of 
Nakazono (Japanese Patent Publication 2003-134385) [hereafter Nakazono]. 

1 0. As to claim 7, it is noted that the combination of the Nonaka, Freeman, 
and Ono references fails to particularly disclose a discriminating unit for 
discriminating different subjects wherein the shake amount obtaining unit obtains 
a shake amount for each of the different subjects and the image combining unit 
combines images for each of the different subjects. 

On the other hand, Nakazono teaches a camera (Fig. 1) that captures an 
image using a CCD imager (1) and using an image composition device (4) that 
comprises a motion vector detecting element (Fig. 2, 11, Fig. 3) that uses two 
pictures to discriminate subject images (Fig. 5) and determines a shake amount 
for the subject using a motion vector calculation part (Fig. 3, 23) and the two 
images used to determine a shake amount of the subject of the image are 
combined (Fig. 15) into an output image using a synthesizing means (Fig. 14, 92) 
(Page 1 1 , 0056-0058, Page 12, 0059-0060, Page 16, 0073-0075). 

It would have been obvious to one having ordinary skill in the art at the 
time of invention to discriminate subjects of an image and obtain shaking 
amounts of the identified subjects as taught by Nakazono with the multi-eye 
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imaging apparatus of Nonaka, modified with the teachings of Freeman and Ono, 
because the prior art are directed towards imaging devices that eliminate image 
shaking effects and because the claimed limitations are disclosed within the cited 
prior art and because obtaining shake amounts, or movement amounts, of an 
imaged subject between two images would allow the device of Nonaka to 
determine an image shake amount for an image's subject as opposed to the 
entire image in order to compensate for an image where the subject itself is 
moving as opposed to the physical camera, and combining the images would 
result in an image free from shaking or blurring effects . 

11. As to claim 8, the Nakazono reference discloses all claimed subject matter 
with regards to similar comments of claim 7. Additionally, Nakazono teaches a 
division unit for dividing image information into a plurality of blocks (Fig. 8) and 
obtaining a shake amount for a plurality of blocks using block setting means (Fig. 
3, 22) (Page 13, 0064-0065). 

12. Claim 9 is rejected under 35 U.S.C 103 as being unpatentable over 
Nonaka et al. (US Patent 7,162,151) [hereafter Nonaka] and Freeman (US 
PGPub 2005/0280714) [hereafter Freeman] and Ono (US Patent 7,456,874) 
[hereafter Ono], as applied to claim 1, in further view of Yamasaki (US 
PGPub 2003/0071905) [hereafter Yamasaki]. 
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1 3. As to claim 9, Nonaka teaches the optical photographing lens (1 1 ) forms a 
main image of a subject (1 1 0) onto an image sensor (1 2), which captures all 
incident light and corresponding colors, and that light receiving lens (16) receives 
reflected luminous flux from the subject and an image sensor (17), which also 
captures all incident light and corresponding colors, is used to form images 
based on the reflected luminous flux and subsequently these images are used to 
determine moving direction of the image sensor (12) (Col. 3, 42-53, 66-67, Col. 
4, 1-8, 20-28). 

It is however noted that the combination of the Nonaka, Freeman, and 
Ono references fails to particularly disclose an imaging system for handling a red 
color, an imaging system for handling a green color, and an imaging system for 
handling a blue color, wherein for at least one corresponding to one color of the 
imaging systems corresponding to the respective colors, the number of the 
imaging systems corresponding to the one color is two or more, and the two or 
more imaging systems for handling the one color include the first imaging system 
and the second imaging system. 

On the other hand, Yamasaki discloses an image sensing unit (Figs. 2-6) 
with a plurality of imaging systems having different optical axes composed of an 
imaging system (aperture 205b, filter 206b and sensing region 204b) for handing 
a red color, an imaging system (apertures 205a and 205d, filters 206a and 206d, 
and sensing regions 204a and 204d) for handling a green color, and an imaging 
system (aperture 205c, filter 206c and sensing region 204c) for handling a blue 
color, wherein for at least one corresponding to one color of the imaging systems 
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corresponding to the respective colors, the number of imaging systems 
corresponding to the one color (green) is two or more (Page 3, 0057-0060) and 
the two or more imaging systems for handling the one color (green) include the 
first imaging system and the second imaging system (Page 4, 0080, Page 5, 
0082-0083). 

It would have been obvious to one having ordinary skill in the art at the 
time of invention to include an imaging system for handling a red color, an 
imaging system for handling a green color, and an imaging system for handling a 
blue color, wherein for at least one corresponding to one color of the imaging 
systems corresponding to the respective colors, the number of the imaging 
systems corresponding to the one color is two or more, and the two or more 
imaging systems for handling the one color include the first imaging system and 
the second imaging system as taught by Yamasaki with the multi-eye imaging 
apparatus of Nonaka, modified with the teachings of Freeman and Ono, because 
the prior art are directed towards imaging apparatuses with imaging systems 
having different optical axes that combine captured image information wherein 
the image information have determined parallax amounts and because all the 
claimed limitations are disclosed within the cited prior art and would enable an 
imaged scene to be captured according to a plurality of different colors where 
each color is handled by a respective imaging system thus expanding the color 
information of the imaged scene capable of being captured by the imaging 
apparatus. 
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Conclusion 

14. Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 
See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Michael Osinski whose telephone number is 
(571) 270-3949. The examiner can normally be reached on Monday to Thursday 
9 a.m. to 6 p.m. If attempts to reach the examiner by telephone are 
unsuccessful, the examiner's supervisor, Jason Chan can be reached on (571) 
272-3022. The fax phone number for the organization where this application or 
proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
9199 (IN USA OR CANADA) or 571-272-1000. 

MO 

/MICHAEL OSINSKI/ 
Examiner, Art Unit 2622 
/Jason Chan/ 

Supervisory Patent Examiner, Art Unit 2622 



